Europaisches Patentamt 
European Patent Office 
Office europ£en des brevets 



III! 

© Publication number: 



0 651 543 A2 



© Application number: 94307981.4 
® Date of filing: 28.10.94 



EUROPEAN PATENT APPLICATION 

© inter* H04M 1/00, H04M 1/274 



© Priority: 01.11.93 US 146341 

© Date of publication of application: 
03.05.95 Bulletin 95/18 

® Designated Contracting States: 
DE FR GB 

© Applicant: International Business Machines 
Corporation 
Old Orchard Road 
Armonk, N.Y. 10504 (US) 

© Inventor: Allard, David Joseph 
3900 Greenfield Court 
Boynton Beach, 
Florida 33436 (US) 
Inventor: Can ova, Francis James, Jr. 
922 S.W. 35th Avenue 
Boynton Beach, 
Florida 33435 (US) 

Inventor: Johnson, Debra Ann Gawne 

3040 N.E. 47th Street 

Fort Lauderdale, 

Florida 33308 (US) 

Inventor: Lanier, Charles Sterling 

305 S.W. 27th Avenue 



Del ray Beach, 
Florida 33445 (US) 
Inventor: Lewis, James Robert 
5376 N.W. 55th Street 
Coconut Creek, 
Florida 33073 (US) 
Inventor: Tiller, Byron Kevin 
9659 Ohio Place 
Boca Raton, 
Florida 33434 (US) 
Inventor: Villafana, William 
2945 S.W. 111th Terrace 
Davie, 

Florida 33328 (US) 
Inventor: Yee, Raymond Lee 
9965 North West 52nd Street 
Coral Springs, 
Florida 33076 (US) 



© Representative: Moss, Robert Douglas 
IBM United Kingdom Limited 
Intellectual Property Department 
Hursley Park 
Winchester 

Hampshire S021 2JN (GB) 



© Personal communicator having improved zoom and pan functions. 



© A mobile, hand-held personal communicator 10 
^ includes a casing 12 for housing a cellular phone, 
^ modem, and data processing system. Graphics im- 

age files 142 are stored and can be selectively 
^J* displayed on a touchscreen display 26, 28. A zoom 
IT) function magnifies areas of a graphics image 142, 
^ m such as a fax image, that has been received and 
ID stored in the personal communicator. When the 

zoom function is selected, the user touches the 
© screen, and a magnification frame 168 or window 
_ appears. When the user removes his/her finger, the 
m area within the frame is magnified. The user can also 



drag the frame 168 around the screen, while still 
touching the screen, to select a specific area to be 
magnified when the user releases the screen. A pan 
function allows the user to shift the image 170 within 
a viewing area 142. The user can pan the image 170 
by touching the display at an initial touch point 188, 
and moving his/her finger keeping it in contact with 
the screen, to shift the touch point to a new touch 
point 190. Upon releasing the new touch point, the 
image is redrawn in a new position corresponding to 
the change in position between the initial 188 and 
new 1 90 touch points. 
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Held of the Invention 

The invention relates to the field of personal 
communicators having a variety of data processing 
and communications functions. More particularly, 
the invention relates to a zoom and pan function for 
use with such a personal communicator. 

Background of the Invention 

The personal communicator is in the form of a 
handset having an earphone or speaker at one end 
and a microphone at the other end. The speaker 
and microphone are spaced apart the same dis- 
tance as the distance between an ear and the 
mouth of an "average" user, to facilitate use of the 
communicator as a cellular phone. A small touch- 
screen display is located between the speaker and 
the microphone. The display has a fine pitch liquid 
crystal display (LCD) allowing it to be operated as 
a standard CGA with a resolution of 640x200 pix- 
els. The touch sensitive display provides a graphic 
user interface and is used as an input/output de- 
vice for a variety of functions or applications. 

Within the prior art, particularly that of desktop 
personal computers, it is publicly known to control 
pan and zoom functions on graphic images by 
positioning a cursor with the aid of a cursor control 
device such as a keyboard, mouse, trackball, pen, 
etc. With a mouse controlled screen, a common 
zoom function is to click at one corner of the area 
to be magnified, and drag the mouse to another 
corner. Upon release, the delineated area is mag- 
nified. Panning is done by moving the cursor 
against the edge of a screen image. U.S. patent 
5,187,776- Yanker discloses an "IMAGE EDITOR 
ZOOM FUNCTION" for a personal computer CRT 
display, in which a small viewport window shows 
an image at a <1x) magnification. The viewport is 
superimposed on a larger display of a magnified 
portion of the image centered about the location of 
the cursor in the viewport. The cursor is also mov- 
able in the viewport to pan the magnified image. 

U.S. patent 4,899,292- Montagan et al, dis- 
closes a "SYSTEM FOR STORING AND RETRIEV- 
ING TEST AND ASSOCIATED GRAPHICS". The 
system includes a computer coupled to a terminal 
having a resistive touchscreen. A wand is con- 
nected to an analog to digital converter. When the 
wand is touched to the screen, it detects a voltage 
signal that is converted into coordinates. The im- 
age is magnified by two centered at the point of 
touch. 

Disclosure of the Invention 

One of the objects of the invention is to provide 
improved zoom and pan functions for a mobile, 



handheld personal communicator having a small 
touchscreen display actuated in response to being 
touched by a user's finger. 

Another object of the invention is to provide a 

5 novel and improved means for selecting areas of a 
touchscreen graphic image to be panned or mag- 
nified, without requiring the use of a pointer device 
such as a trackball, pen or mouse. 

A further object of the invention is to provide a 

10 personal communicator in which graphic images 
are stored for display on a finger operated touch- 
screen display, and a displayed image can be 
zoomed by touching the display whereby a frame 
appears delineating the area to be magnified, the 

75 frame being dragged by sliding the finger across 
the display, until the frame indicate the area the 
user wants magnified. 

Briefly, in accordance with the invention, a 
zoom function is used to magnify areas of a graph- 

20 ics image, such as a fax image, that has been 
received and stored in the PC. When the zoom 
function is selected, the user touches the screen, 
and a magnification frame or window appears. 
When the user removes his/her finger, the area 

25 within the frame is magnified. The user can also 
drag the frame around the screen, while still touch- 
ing the screen, to select a specific area to be 
magnified when the user releases the screen. 

When the pan function is selected, the entire 

30 screen acts as a window or mask over the image. 
The image thus appears as a sheet of paper be- 
hind the screen. The user can then press the 
screen anywhere at an initial touch point, and move 
his/her finger keeping it in contact with the screen, 

35 to shift the touch point to a new touch point. Upon 
releasing the new touch point, the image is re- 
drawn in a new position corresponding to the 
change in position between the initial and new 
touch points. 

40 

Brief Description of the Drawings 

This invention will now be described, by way of 
example only, with reference to the accompanying 
45 drawings, in which: 

Rg. 1 is a plan view of a personal communicator 
embodying the invention; 

Fig. 2 is a longitudinal view through the personal 
communicator shown in Fig. 1, schematically 
so illustrating various components; 

Fig. 3 is a is a block diagram of electrical 
components of the personal communicator 
shown in Fig. 1; 

Fig. 4 is a illustration of a sequence of screens, 
55 beginning with the first screen that appears 
when the PC is turned on, showing how to get 
into zoom and pan functions; 
Rg. 5 is a flow chart of the zoom function: 
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Fig. 6 is a diagram of different screens illustrat- 
ing the zoom function; 

Fig. 7 is a flow chart of the pan function; and 
Fig. 8 is a diagram of different screens illustrat- 
ing the pan function. 

DETAILED DESCRIPTION 

General System Organization 

Referring now to the drawings, and first to Fig. 
1, a personal communicator (PC) 10 is of the type 
generally set forth above and comprises a cellular 
phone and a complete data processing system 
further details of which are described below. PC 10 
includes a housing or casing 12 having mounted 
along one side thereof a plurality of external, man- 
ually operated push buttons including an ON/OFF 
push button 14, an UP push button 16, and a 
DOWN push button 18. PC 10 has a size facilitat- 
ing usage as a hand-held device and buttons 16 
and 18 are located so that a right handed user can 
wrap his/her fingers around the rear of the casing 
and actuate the buttons to adjust speaker volume 
while the user is listening to the phone speaker. A 
pull-out antenna 20 is mounted at the top of the 
casing. A speaker 22 is mounted near the top of 
the casing and a microphone 44 (Fig. 2) is moun- 
ted inside the bottom of the casing behind port 
holes 24. A touchscreen display 26 is mounted on 
the middle portion of the casing between the 
speaker and the microphone, and provides an in- 
put/output (I/O) device and user interface for enter- 
ing information into and receiving information, from 
the PC and the data processing system therein. 
The general shape of PC 10 is similar to the shape 
of commercially available portable phones except 
that display 26 occupies the space normally used 
for a standard, electromechanical, twelve push-but- 
tons, phone keyboard. Display 26 includes a touch 
sensitive overlay 28 the outer surface of which is 
flush with adjacent portions of casing 1 2. 

Referring to Fig. 2, PC 10 has a plurality of 
miniaturized components mounted within casing 
12. The components include speaker 22 and micro- 
phone 44 spaced apart a distance corresponding to 
the distance between an ear and the mouth of an 
average size user to facilitate use as a phone. PC 
10 is battery operated and includes a battery 46 
detachably mounted on the back wall of the casing. 
A cellular phone transceiver 48 is mounted inside 
casing 12 and is connected to antenna 20, the 
antennal being extensible outwardly from the cas- 
ing for radio frequency (RF) communications. Dis- 
play 26 is rectangularly shaped and comprises a 
touch sensitive overlay 28 that overlies a liquid 
crystal display (LCD) 38, and a backlight 40. The 
LCD has a viewing area of 36mm by 115mm and is 



operated as an all points addressable display. A 
planar card 50 is mounted within the center of 
casing 12 and has mounted thereon various com- 
ponents including a power supply 52, a central 

5 processing unit (CPU) 54 that forms the system 
controller (Fig. 3), a modem 56, and other compo- 
nents 58 including a connector for connecting and 
supporting an optional Personal Computer Memory 
Card International Association (PCMCIA) card 57 

io that conforms to published standards of the Associ- 
ation for a version 2.0 card support. 

Referring to Fig. 3, system controller 54 is 
single chip implementation of a PC/XT class porta- 
ble computer, the chip being a Model VG230 pro- 
fs cessor commercially available from Vadem, Inc., 
San Jose, Cal. Controller 54 comprises a plurality 
of subsystems including a microprocessor 60 for 
executing various programs stored in the commu- 
nicator, an LCD controller 62, a COM1 communica- 

20 tion port 64, and pinouts for connection to various 
devices the major ones of which are shown in Fig. 
3. A bus 66 interconnects controller 54 with a main 
memory 68, a read only memory (ROM) 70, a disk 
ROM 72, touch controls 74, and modem 56. 

25 ROM 70 stores a basic input/output system 

(BIOS) 76 and a power on self test (POST) pro- 
gram 78, and is accessed in the usual manner. 
Disk ROM 72, however, is accessed, in a manner 
that simulates a disk drive, and stores files contain- 

30 ing a disk operating system (DOS) 79, the Naviga- 
tor program 80 which is an extension to the DOS, 
application programs 81 that produce various com- 
munication and data processing functions in PC 10, 
and interrupt handlers 83 for handling interrupts 

35 specific to the applications. Additional memory and 
application programs can be added by the 
PCMCIA card. When the communicator is turned 
on, POST program 78 is executed. In response to 
a successful completion of the test, a kernel of 

40 DOS 79 is then loaded from disk ROM 72 into 
main memory 68, for execution. DOS 79 accesses 
disk ROM 72 and loads the Navigator and the 
application programs, as needed, into main mem- 
ory 68 for execution by microprocessor 60. The 

45 Navigator controls the information displayed on the 
LCD, paints the various screens associated with the 
application programs on the LCD, and spawns new 
programs in response to user selection through 
actuation of the touchscreen interface. 

so Communication functions are handled by RF 

deck 48 and modem 56. RF deck is connected to a 
beeper 94 that is sounded or actuated to alert the 
user to an incoming call. Controller 54 is also 
connected to beeper 94 and is able to selectively 

55 actuate the beeper such as when a screen button 
is selected. Deck 48 is also connected to an audio 
mux 96 which controls the routing of audio signals 
between deck 48, speaker 22, microphone 44, 



4 



5 



EP 0 651 543 A2 



6 



modem 56, and audio port 98. Modem 56 is con- 
nected to a phone port 1 00 by means of which PC 
10 can be connected by a cable (not shown) into a 
phone network or system. Deck 48 is also con- 
nected to a mux 102 that in turn is connected 
further to a serial port 104 and to COM1 port 64 of 
controller 54. 

Battery 46 is connected to power supply 52 
and supplies the primary power for operating PC 
10. A backup battery 82 is also connected to the 
power supply and supplies power to volatile mem- 
ory when the main battery is removed for replace- 
ment. Power supply 52 is connected to a backlight 
inverter 84 that, in turn, is connected to backlight 
40 for controlling back lighting of the LCD to allow 
PC 10 to be operated under low light conditions. 
PC 10 preferably includes power management 
functions (not shown) for conserving energy usage 
and prolonging battery life. 

LCD controller 62 is connected by a data bus 
88 to supply data to LCD 38 to control the contents 
of the information displayed by the LCD. Controller 
62 drives a contrast control signal (CCS) onto line 
90 for transmission to power supply 52. In re- 
sponse to the CCS, power supply 52 drives a 
contrast voltage signal (CVS) onto line 92 that 
determines the contrast of LCD 38. 

Touch overlay 28 is a resistive pressure sen- 
sitive overlay that produces an analog voltage sig- 
nal indicative of an overlay location being pressed. 
Overlay 28 is connected to touch controls 74 that 
convert the analog signal into a digital signal repre- 
senting the X,Y coordinates of the overlay location 
being pressed. The digital coordinates are then 
transmitted over bus 66 to controller 54 and mem- 
ory 68 for use in determining the purpose of the 
location being pressed. Touch controls 74 are fur- 
ther connected to multifunction logic 86. In re- 
sponse to sensing that the overlay 28 has been 
touched, controls 74 generate an interrupt request 
that is transmitted to logic 86 and logic 86 then 
transmits the interrupt request to controller 54. UP 
and DOWN buttons 16 and 18 are also connected 
to logic 86 which debounces any signals due to the 
mechanical nature and generates a button number 
identifying which button has been pressed. The 
numbers are stored in I/O ports (not shown) which 
can be polled by the system. Multifunction logic 86 
is also connected to RF deck 48 and audio mux 96 
allowing controller 54 to communicate therewith 
and provide functions not done by the system 
controller, such functions not being germane to the 
invention. 

Operation 

As indicated above, when PC 10 is turned on, 
DOS 79 and Navigator 80 are loaded into main 



memory 68, along with an application. By default, 
the first application to be loaded is a phone ap- 
plication that provides and controls cellular phone 
operations. Such application is executed and inter- 

s acts with the Navigator program to create a Phone 
screen 108 on LCD 38, which screen is shown in 
Fig. 4. The various screens displayed on LCD 38 
are painted by Navigator 80 using information from 
the specific application program to which the 

70 screen is relevant. 

The screens include selectable buttons having 
thin solid lines along their left and top edges and 
shadow lines along their bottom and right edges. 
The buttons also bear labels or icons, or both. A 

75 button is selected by a touch and release action. 
That is, a button is selected when the user touches 
overlay 28 in the area overlaying the button, and 
then releases the button by releasing the overlay. 
When a button is so touched, it is highlighted. 

20 When a button is so selected, the system sounds 
beeper 94 to provide audio feedback to the user. 
All buttons are also operated by a "last contact" 
method so that a user can easily recover from 
touching an unintended button by leaving a finger 

25 on the screen and sliding to another button or a 
non-button area. As a finger is slid across the 
overlay, new buttons are highlighted while the old 
buttons are returned to normal, non-highlighted 
conditions, thereby providing the user with visual 

30 feedback as to which button will be selected when 
the finger is removed from the overlay. Each ap- 
plication program stores one or more maps cor- 
relating the X.Y positions of the overlay with the 
different buttons and applications. The Navigator 

35 scans the map, in response to receiving X,Y co- 
ordinates, to determine the function of a selected 
area or button, and take appropriate action. 

Screen 108 includes a simulated cellular phone 
keyboard 120 and a data entry area 122 in which 

40 phone numbers are displayed as the respective 
digits of a phone number are entered by the key- 
board. Screen 108 also comprises a row 111 of 
selectable buttons including a HELP button 112, a 
PHONE button 114, a TOOLS button 116, and a 

45 BACK button 118, located along the bottom edge 
of the screen. These four selectable buttons are 
common to other menu screens and appear at the 
same locations on screen 110 and other portrait 
screens. PHONE button 114 is used to transfer 

so control to the phone application from other applica- 
tions, to allow the user to place or receive a phone 
call regardless of the current application. TOOLS 
button 116 is used to transfer control to an office 
tools screen allowing the user to select another 

55 application. BACK button 118 is used to transfer 
control back to the previously selected screen(s). 

When TOOLS button 116 of screen 108 is 
selected, the Navigator then paints a Tools screen 
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124, where arrow 126 represents switching from 
Phone screen 108 to Tools screen 124. Tools 
screen 124 displays a plurality of selectable but- 
tons 128 that allow a user to select an application 
from a menu of different applications such as an 
address book, a calculator, a FAX, E-mail, data 
base services, and a note pad. When the FAX 
button is selected, the Navigator switches to the 
Fax application via arrow 129, and a FAX screen 
1 30 is painted on display 38. 

FAX screen 130 displays a plurality of selec- 
table buttons 132 that form a list of different fax 
files stored in the system. Buttons 134 are for the 
purpose of scrolling down and up in the list. Row 
111 of the common buttons, is displayed at the 
bottom of screen 130. In response to the selection 
of one of buttons 132, the Navigator switches, at 
140, to a FAX VIEW screen 138 having a view area 
142 within which the first page of the selected fax 
file, is displayed. Displayed at the bottom of screen 
138, beneath view area, are: a row 111 of common 
bottoms; two rows 144 of buttons for selecting 
functions of Save, Quit, Erase, Rename, and Un- 
Zoom; and a row 146 of tool buttons Draw, Eraser, 
Zoom, and Pan. 

By way of example, suppose a fax file named 
DIAGRAM contains an image of a flow chart similar 
to Fig. 5 of the drawings herein. In response to 
selecting the button 132 with the name DIAGRAM 
therein, the Navigator and fax application cause the 
illustrated image 143 to appear in the viewing area 

142 of FAX VIEW screen 138. The initial image 

143 represents the full, first page of the file with no 
magnification. Quite obviously, the image, in this 
example, is so small that details thereof are not 
perceivable by a user. The zoom and pan functions 
described below allow the user to magnify the 
image and shift it around, as desired. 

The zoom function is initiated or started when 
the user selects the Zoom tool 1 46 at the bottom of 
screen 138. With reference to Figs. 5 and 6, when 
the Zoom tool is selected, a zoom mode of opera- 
tion is begun at 150. Initially, a decision 152 is 
made as to whether or not, the touchscreen is 
being touched. If not, a loop is made until the user 
touches the screen at a touch point, whereupon a 
rectangular highlighted box 168 is drawn with the 
touch point being within the borders. Box 168 de- 
lineates the area of viewing screen 142, which is to 
be zoomed or magnified. Preferably, the shape of 
box 168 is similar to but smaller than the shape of 
viewing area 142. Next, a decision is made at 156 
whether or not the touch point has been released. 
If not, a loop is entered with decision 162 and 
continues until the loop is broken the releasing the 
touch point, such as by removing the user's finger 
from the display, whereupon step 158 displays the 
portion of the image within box 168 to fill the 



viewing area with an enlarged image 170 having 
the same aspect ratio. At that point, the zoom 
mode ends and a return is made at 160 to the 
normal pan view screen, which at that point contin- 

s ues to display image 170. rf further magnification is 
desired, the user can repeat the process. Decision 
162 determines if the touch point has changed 
which would be caused by sliding the finger over 
the display to a new position. If a change occurs, 

10 step 164 moves the highlighted box to a new 
position around the new touch point. The loop is 
traversed as the user moves the touch point, until 
the finger is removed from contact with the touch- 
screen display. In this manner, the user is able to 

75 select the specific area desired to be magnified. 

Relative to the exemplary series of screens 
shown in Fig. 6, the size of box 168 relative to the 
size of view area 142 produces a magnification of 
two. Obviously, other ratios can be used. If the user 

20 touches the center of the view area, the box 1 68 is 
centered on the point of touch. But if the user 
touches the view area near an edge of the view 
area, the box is shown in full and would not be 
centered at the point of touch. Beginning with the 

25 left screen in Fig. 6, the user touches the screen at 
point 166, and box 168 is drawn. The box initially 
may not fully surround, diagram 143. Then, the 
user moves the finger until box 168 contains image 
143, as shown in the center screen. Upon removal 

30 of the finger, the view area is filled with a magnified 
image 170. 

The pan function is initiated or started when 
the user selects the Pan tool 146 at the bottom of 
screen 138. With reference to Figs. 7 and 8, when 

35 the Pan tool is selected, a pan mode of operation 
is begun at 172. Initially, a decision 174 is made as 
to whether or not, the touchscreen is being 
touched. If not, a loop is made until the user 
touches the screen at a touch point such as at 

40 point 186, whereupon a small rectangular box 188 
is drawn with the initial touch point being within the 
borders of the initial box 188. A loop is formed with 
decision 180 being made in response to a negative 
decision out of 178. Decision 180 decides if the 

45 touch point position has changed. If so, a new box 
190 is drawn and is connected to box 188 by a 
rubber-band 192. As the finger is slid around the 
view area, box 190 is constantly redrawn beneath 
the finger, and the rubber-band is constantly re- 

50 drawn to give the appearance of stretching and 
contracting. The finger is moved in the direction 
and to the same extent that the user wants to shift 
or move the image. Upon release of the screen, 
decision 178 passes control to step 184 which, in 

55 response to releasing the screen, points or redraws 
the image 170 in a position that is displaced from 
the initial position in the same direction and to the 
same extent as the displacement of box 1 90, at the 
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time of release, from initial box 188. The process 
can be repeated to shift the image to a new posi- 
tion. It may be that portions of the initial image are 
lost when shifted outside the view area, and/or that 
new image portions come into view. The combina- 
tion of the pan function with the zoom function 
allows a user to readily view different portions of a 
full page of fax material on the small display. 

As should be apparent to those skilled in the 
art, the programs are stored as electrical signals in 
the different memories. The term "processing 
means " as used in the claims encompasses not 
only memories which store the programmed sig- 
nals but also the different components, including 
the microprocessor, which operate in response to 
the programmed signals, to execute the different 
processing functions and operations described 
above. 

Claims 



redrawing a second symbol (190) beneath the 
finger as the finger slides to other positions; 
and 

eighth means for redrawing said graphics 
5 image in a new position in response to a 

release of said display at said second symbol. 

4. A personal communicator (10) as claimed in 
claim 1 wherein said frame (168) and said 
w touch sensitive display (26, 28) have similar 

shapes, said frame having a size smaller than 
that of said touch sensitive display and provid- 
ing a predetermined magnification ratio. 

75 5. A personal communicator (10) as claimed in 
claim 1 wherein said frame (168) and said 
touch sensitive display (26, 28) have rectangu- 
lar shapes, said frame being displayed on said 
display beneath the point on said overlay 

20 touched by the finger. 



1. A personal communicator (10) comprising: 

a casing (12) having a size adapted to be 
held in the hand of a user; 

a touch sensitive display (26, 28) mounted 25 
on said casing; 

first means for displaying a graphics im- 
age (143) on said display; 

second means for displaying a magnifica- 
tion frame (168) on said display in response to 30 
a user touching said display at a touch point 
(166) with a finger, said frame being located 
beneath said touch point and delineating a 
portion of said graphics image to be magnified; 

third means for moving said frame around 35 
said display in response to sliding said finger 
around said display and in contact therewith; 
and 

fourth means for magnifying said portion of 
said graphics image delineated by said frame, 40 
rn response to disengaging said finger from 
said display. 

2. A personal communicator (10) as claimed in 
claim 1 wherein said second means is selec- 45 
tively actuated, and said communicator further 
comprises selectively actuated fifth means for 
panning said graphics image (143) in response 

to said user touching said display with a finger 
at one point and sliding the finger to another 50 
point thereby indicating direction and extent of 
panning. 

3. A personal communicator (10) as claimed in 
claim 2 wherein said fifth means comprises: 55 

seventh means for drawing a symbol (188) 
on said display indicating where the user ini- 
tially touched said display and for continuously 
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